arXiv:math/0512393vl [math.PR] 16 Dec 2005 


A Universal Dilation of Discrete Markov Evolutions 


M. Gregoratti* 


Dipartimento di Matematica “F.Brioschi”, Politecnico di Milano 
Piazza Leonardo da Vinci 32, 1-20133 Milano, Italy 


Quaderno di Dipartimento n. 649/P 
December 15, 2005 


Abstract 

Given a finite state space E, we build a universal dilation for all possible discrete time 
Markov chains on E, homogeneous or not: we introduce a second system (an “environment”) 
and a deterministic invertible time-homogeneous global evolution of the system E with this 
environment such that any Markov evolution of E can be realized by a proper choice of 
the initial (random) state of the environment, which therefore determines the transition 
probabilities of the system. We also compare this dilation with the quantum dilations of a 
Quantum Dynamical Semigroup: given a Classical Markov Semigroup, we show that it can 
be extended to a Quantum Dynamical Semigroup for which we can find a quantum dilation to 
a group of *-automorphisms admitting an invariant abelian subalgebra where this quantum 
dilation gives just our classical dilation. 
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1 INTRODUCTION 

A well known theorem due to Birkhoff shows that any doubly stochastic matrix is a convex com¬ 
bination of permutation matrices. These describe with matrix terminology one-step evolutions 
which are both deterministic and invertible. Analogously, a theorem due to A. S. Davis [3] shows 
that any stochastic matrix, doubly stochastic or not, is a convex combination of deterministic 
matrices. These describe with matrix terminology one-step evolutions which are deterministic 
but not necessarily invertible. As a consequence, A. S. Davis shows that each (finite state) 
Markov chain can be realized as an automaton with random inputs, thus establishing a connec¬ 
tion between Markov chains and automata theory. 

The relationship between Markov chains and deterministic dynamics can be further ana¬ 
lyzed looking for realizations of Markov evolutions as deterministic invertible time-homogeneous 
evolutions of the system coupled with a second system. Such representations are usually called 
dilations and they are important to show that a Markov evolution of a given system is com¬ 
patible with a deterministic invertible homogeneous model for the evolution of a bigger system. 
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Thus the stochastic features of the system evolution are given a dynamical explanation. This is 
theoretically relevant if Markov chains are applied to phenomena, like physical phenomena for 
example, for which an underlaying theory postulates deterministic invertible homogeneous evo¬ 
lutions in absence of noise and external disturbances. For these phenomena the second system 
introduced by the dilation models the surrounding world, the environment, which is the source 
of the noise. 

Dilations are indeed typical in Quantum Probability, where the cultural influence of Quan¬ 
tum Physics always emphasized the importance that a stochastic evolution of a (non isolated) 
quantum system does not contradict the axioms of Quantum Mechanics, i.e. that it can arise 
form a unitary homogeneous evolution of a (isolated) bigger quantum system, consisting of the 
given system and its environment. 

Here we are interested in the classical analogue of quantum dilations. More precisely, in 
a completely classical framework, we consider a system with finite state space E, undergoing 
a discrete time evolution given by a Markov chain, not necessarily homogeneous. Then, in 
order to get a dilation of this chain, a second system is introduced, called environment, with 
its state space (T, Q), a measurable space, together with a global invertible one-step evolution 
a : E xT —> E xT. Thus, if (i,y) is the state of the compound system at time 0, then a*(i, 7) 
is its state at time f £ N, where hence a * gives a deterministic invertible homogeneous global 
evolution. If, nevertheless, the state of the environment is never observed and if initially it is 
randomly distributed with some law Q on (r,f?), the evolution of the observed system turns 
out to be stochastic and, if T, Q, a and Q are properly built, it is given by the original Markov 
chain. In this case, we say that (r, Q, a, Q ) is a dilation of the Markov evolution on E. 

In this paper, given only the state space E (arbitrary but finite), we build a universal 
dilation (T,Q,a,{Q}), where {Q} is an entire family of distributions which can produce any 
Markov chain on E: every Markov chain, homogeneous or not, can be dilated by taking always 
the same, universal, (T, Q, a) and by choosing every time the proper distribution Q for the initial 
state of the environment. Moreover, our construction, which relays on A. S. Davis decomposition 
of stochastic matrices, allows to interpret each Markov chain, not only as the stochastic dynamics 
resulting from the coupling with an environment, but at the same time also as an automaton 
with random inputs, which are now dynamically provided by the environment itself. 

An unpublished result by Kiimmerer [6] provides a dilation of a Markov chain with a con¬ 
struction similar to ours, but it does not exhibit the same universality because the interaction 
a depends on the chain under consideration. Our aim is similar also to the aim of Lewis and 
Maassen [8] when they consider classical mechanics in continuous time and, taken a linear Hamil¬ 
tonian system modelling a particle and its environment, they describe how Gibbs states of the 
whole system lead to stationary Gaussian stochastic processes for the observables pertaining to 
the particle under consideration. However, we do not look for good global states, but for good 
states Q of the environment alone which lead to Markov evolutions of the system E, our particle, 
for every independent choice of its initial state. 

The paper also shows the relationship between a universal dilation (T, Q, a, {Q}) and the 
quantum constructions which inspired it. In Quantum Probability [4,6,7,9-11] the starting 
object is a Quantum Dynamical Semigroup (QDS) which describes the evolution of a non isolated 
system. It is the quantum analogue of a Classical Markov Semigroup (CMS). Then a dilation 
realizes the QDS as a group of ^-automorphisms describing an invertible homogeneous evolution 
of the system coupled with an environment or, equivalently, as the expectation of a Quantum 
Stochastic Flow, which is a quantum generalization of a Markov stochastic process. We shall 
prove that our dilations (T,Q,a,Q) are classical restrictions of quantum dilations: every CMS 
on E admits an extension to a QDS for which we can find a quantum dilation which is itself 
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an extension of the dilation of the CMS. However, we shall not embed a universal dilation 
(T,Q,a,{Q}) in the quantum world, as quantum dilations do not exhibit the same universality 
and they strictly depend on the QDS under consideration, so that it is not enough to change 
the environment state to get another QDS. 

The paper is divided into two parts. The first part is completely classical and it consists 
of Section 2, where we set up the framework, and of Section 3, where we give the definition of 
universal dilation of Markov evolutions on E and we show that there exists always one by a 
constructive proof. Then the second part, Section 4, is devoted to the set up of the quantum 
framework and the analysis of the relation between our (classical) universal dilation of a CMS 
and the (quantum) dilation of a QDS. 


2 PRELIMINARIES 

We consider a system with finite state space E = (1,..., N}, fixed for the whole paper. We 
denote by P = ( Pij)i,j£E a stochastic matrix on E, so that pij > 0 and J2jPij = 1 for every i, 
and we identify the elements of the (complex abelian *-) algebra C°°(E), the system random 
variables / : E —> C, with the column vectors in C N , so that every stochastic matrix P on E 
defines an operator on C°°(E), 

(Pf )(*) = ^PijfU)’ 

jeE 

describing a one-step probabilistic evolution. Taken a sequence of stochastic matrices (P(t)) t£ ^, 
the evolution of a system random variable / from time 0 to time t > 1 is therefore given by 

f^f t :=P(l)...p(t)f, V/ G £°°(E). (1) 

If the sequence is constant, P{t) = P for every t, then the evolution is homogeneous and it is 
described by the Classical Markov Semigroup (-P*)t>o- 

We denote by D a deterministic matrix on E, a stochastic matrix with a 1 in each row. 
Every D describes with matrix terminology a deterministic (not necessarily invertible) evolution 
P, where 

D = (, P : E * E, dij = (2) 

so that Df = f o p. The invertible (bijective) maps on E correspond to the special cases of 
permutation matrices. The deterministic matrices are just the extreme points of the convex set 
of stochastic matrices and every P is a convex combination of deterministic matrices, 

P = £>£><, Qe > 0, = 1- (3) 

i&L i&L 

One can find such a decomposition with N n terms, with the set L labelling all possible deter¬ 
ministic matrices and weighing each Dp with qi = Pifam ■ • • Pn/3AN )■ Let us remark that the 
decomposition m is not unique and that, for any given P, no more than N 2 — N + 1 terms are 
needed [3]. Anyway, since we are not going to fix P, we shall employ Q in the described form: 
a sum of N n terms which can produce any P simply by changing the weights (qi)e£L- 
For every sequence of stochastic matrices (P(t))t£ n, there exists a Markov chain 
(fl, E, ( Pt)t>o, (Xt)t> 0 ) (Pfc)jfceE) with transition probabilities given by P(t), i.e. a discrete time 
stochastic process of random variables X t : Q ^ E. adapted to a filtration (Et)t>o, and a family 
of probability measures Ffc, k £ E, such that the starting distribution of the process depends on 
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k. Xq has Dirac distribution 5k under P&, but the process always satisfies the Markov property 
with transition matrices P(f): 

¥ k {X t+ i = j\F t ) = PfcPQ+i = j\ x t) = PXtj(t + 1), vk,j,t. 

Chosen a starting distribution 5k, the Markov process law is then uniquely determined by the 
transition matrices P(t). Moreover, a system random variable / E C°°(E ) has now a stochastic 
evolution described by the *-unital homomorphism 

j t : C°°{E) -»• f^jt[f]:=f(X t ), t> 0, (4) 

and the evolution © admits the representation 

ft(k) = (p( 1) • • • P(t) f) (k) = [f(X t )\, V/ € (5) 

and we say that the system has a Markov evolution. 

Let us briefly show a realization of (O, T, (Et)t>o, { x t)t> o, (P k)keE) by means of the decom¬ 
position ©. If P(t) = J2eeL Qt(t) Dii then one can take 

fl = E x L n , uj = £ 2 , ■ ■ •)) P = cylindric c-held, T t = a{{i,l\,... ,£t}',i £ E,£ n E L), 

t 

X 0 (u) =i, X t (u) = Pi t o ■ ■ ■ o fail), P = 4fcII^*( s )> 

3=1 

where {i, l \,..., It} is the event {w = (i, l\, £ 2 , ■ ■ ■, 4 4i-i> ■ ■ ■)> £ L,s > t}. In this way 

the Markov chain associated to (P(f))teN is represented as an automaton with independent 
random inputs: at every step the set of all possible mappings Pe : E —► E is available and the 
system evolution is determined by the value of the input parameter £ which is selected randomly 
according to q(t ) and independently of the previous steps. One could also explicitly introduce 
the random input parameters 

Yf.n^L, Y t (u) = 4 t E N, 

which are clearly independent under every P&. Then 

Pt = a{X 0 , Y„l<8< t), X t = p Yt (X t - l). 

If the sequence P(t ) is constant, then the Markov chain is homogeneous and the input parameters 
can be chosen identically distributed. To get a dilation, we shall introduce a bigger D, namely 
E x G z , with a G bigger than L in order to define an invertible one-shot coupling ip, and with 
Z instead of N in order to get an invertible dynamic with group properties a t . 

In the sequel, given a function / on a domain E , we shall always denote with the same 
symbol / also its trivial extension to a domain BxT, f{i, 7 ) = f(i). 

3 UNIVERSAL DILATION OF DISCRETE MARKOV EVO¬ 
LUTIONS ON A FINITE STATE SPACE 

Let us denote simply by {P} a sequence (P(t))t e pj of stochastic matrices on the state space E 
and let us denote by V the set of such sequences. We call universal dilation of the Markov 
evolutions on E a term 

(R P 1 {Pt)t> 0 , (Zt = ( X t 1 U)) t>0 > 0^k,{P})keE,{P}ev) 1 (6) 

where 
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• (Fl,X, (^i)t>o,P fc ,{P}) is a filtered probability space for every k and {P}, 

• every Zf = (Xf, Tf) is a random variable with values in E x T, being (r,C7) a fixed 
measurable space, 

• for every choice of k and {P}, Xq has distribution 6k, independently of {P}, while To has 
distribution independent of k, say Q{p}j 

• for every choice of {P}, the term (fi, X, (Pt)t>o, (Xt)t>o, (I I> fc,{p})fce-E) is a Markov chain 
with transition matrices {P}, 

• there exists an invertible measurable map a: ExF-^ExF such that Zf = cd(Zo) for 
every t > 0. 

Thus, besides the system E, a second system is introduced, an environment with state space 
Their states Xf and Tf are asked to be random variables on a same measurable space 
(fi, X) such that the global state Z t = (Xf, T t ) undergoes a deterministic invertible homogeneous 
evolution a*. Therefore all the Xf and Tf are determined by Zq, so that Xf and Tf are measurable 
with respect to cr(Zo) = cr(Xo, To) C X and, depending on the probability chosen on X, they 
are deterministic if and only if Zq is. The probabilities Pfc,{p} actually fix the value of Xo but 
usually not that of To- They depend on two indexes, k which fixes the value of Xo, and {P} 
which fixes the distribution of To and the resulting properties of the X t . The space (fi,X) is 
also endowed with a filtration Xt- Note that only the X t are assumed to be adapted to Xt so 
that, in particular, To does not have to be .Po- measura ble. Therefore the Xt are not trivially 
Po- m easurable, even if their values are completely determined by the values of Xo and To, and, 
neglecting the environment, each (II, X, (Pt)t>o, (Xt)t>o, Pfc,{P}) can be a process with a proper 
stochastic evolution. What we ask is that (f2, X, (Xt)t>o, (Xf)*>o, Pjt,{p}) actually is a Markov 
chain starting from k with transition matrices {P}. At the same time however, such a dilation 
shows that this Markov evolution of the system E is compatible with a deterministic, invertible 
and homogeneous model for the evolution of E coupled with an environment T. In particular, as 
Xo = k, the whole stochasticity of the process is due only to the randomness of the unobserved 
environment initial state To- 

We call universal such a dilation because the same (, Xt)t>o, (Zt = (Xf, T t )) i>0 ^ allows 

to represent all the Markov evolutions on E with the change of the probability Pfc,{p} alone. 
Therefore, both the environment state space (T, Q) and the global evolution a do not depend 
on the Markov chain to be dilated, but only on the state space E. 

A universal dilation gives another interpretation of every evolution 0 , compatible with the 
representation 


ft(k) = (P( 1) • • • P(t) /) (k) = E fc>{P} [/(Zf)] = E fc , {P} [f(a\k, T 0 ))] , V/ € C°°(E). (7) 


The stochastic evolution of a system variable / E C°°(E) is now described by / e-> jt[f] '■= 
/(Xf) = /(Zf) = / o (T(Zo). And we can also consider global random variables F :BxT-> C 
and their evolution P i— > P(Zf) = F o a/(Zo). 

In order to show that every state space E admits a universal dilation, now we consider 
a particular class of universal dilations. Let us describe all the special requirements we are 
interested in. 

First of all we want the sample space II to be just ExF, the state space of the global system. 
As we want it to describe all the possible initial global states, we ask the random variable Zo 
to be the identity function and Xq and Yq to be the coordinate variables: if uj = (i, 7 ), then 
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Zq(co) = co, X 0 (co) = i and To(a>) = 7 . Thus, for all t > 1, Z t = (. X t , Y t ) = a* o Zq = Zq o a*, 
X t = Xq o cd and T t = To o ah 

We are interested in an environment T = = X n gGn with G n = G hnite set. In this 

case the environment state 7 £ T has infinite components g n £ G n , n £ Z, and we introduce also 
the coordinate variables Y m (co) = g m , the random m-th components of the environment. Then, 
considered the power cr-held £ on E and the power cr-held on G, we ask Q to be the cylindric 
cr-held on T, and X to be £ < 8 > Q = cr(Ao, To) = <j{Xq, Y n , n £ Z) on 17, so that Xq, To and the 
Y n are all measurable. 

Supposing that at time 0 only Xq is observed and that at each following instant t only the 
information carried by Y t is acquired, we want the filtration Xq = u(Xq), Xt = <j(Xo,Y s ,1 < 
s < t), for t > 1. Coherently we also want an evolution a such that each Yt is involved in 
the interaction with the system only once, between time t — 1 and time t, so that actually 
Xt = b(X t -i,Y t ) for a suitable function b : E x G —> E. In this case Xt is automatically 
adapted to Xt- This is possible if we introduce an invertible map cp\ on E x G 1 , denoted 
by ipi(i,g) = and the invertible shift 1 ? on T = X ne zGn mapping each 

7 = (dn)n to the environment state $( 7 ) with each ra-th component equal to g n + 1 - Then, 
trivially extended cp\ and 1 ? to 17, it is enough to set a = 1 ? o tpi , getting b = ipf. Indeed, 
when a is applied for the first time between time 0 and time 1, the map cp\ couples the system 
state Xq with Y\, giving the new system state X\ = Xq o a = Xq o ipi = ipf (Xq,Y\), and 
then the shift i) prepares Yq for the following interaction with X\. Thus i) could be interpreted 
as a free evolution of the environment. If we explicitly introduce also the random variables 
Yn ' 1 = o q ! , the n-th environment components at time t, then at time 1 the environment 
state is Ti = (Y^) n with (Xq,Y\) and with Y„'^ = Y n+ \ for all n / 0. After t steps, 

X t = <pf (X t -i,Y t ) = <pf (Xt-^Yf-V), while Y 0 (t) = <pf(X t - 1 ,Yl t - 1) ) and Y n (t) = Y^ ] for all 
Ejf 0. 

Finally, we require the probabilities I\. rp} to be 5k <Z> Q{p} on X = £ ® Q, with Q{p}, 
the distribution of To, such that (Y n ) n <o, Y\, Y 2 ,... are independent. Then this independence 
guarantees the Markov property with respect to Xt for the process X t . 

A universal dilation like this will be called standard in the following. Summarizing, a 
universal dilation is standard if 

• T = G* = X nG z G n , G n = G hnite set with power <r-held, Q = cylindric cr-held on T, 

• 17 = E x T, co = (i, 7 ) = (i, (g n ) n ) € fl, i £ E, 7 £ T, g n £ G, 

• Ao(w) — i , To — (kn)nGZ) Y n . 17 > G, To(w) — 7> Y m (co ) — g m > Zq(co~) — co, 

• X = <j{Xq, T 0 ), X t = a(X 0 ,Y s ,l < s <t), t > 0, 

• P^/p} = 5k < 8 > Q{p}, with such that (Y n ) n <o, Y\, Y 2 ,... are independent, 

• a = ’dotp-i, with an invertible map <pi : ExG\ —> ExG\, where tpi(i, g) = ((pf (i, g),ipf (i, g)), 
and the invertible shift $ : T —» T, 

(5 , n)nGZ £ T, g n £ G n (fi f n+l)n€Z £ T, g n +1 £ G n . 


Then, we have 


• Z t = a 1 o Zq = Zq o = (X t , Y t ), 
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T t = T 0 °a t = (Y^) neZ , 


Y. 


(t-i) 


YS^ =Y n oa t = l n+1 (t _ 1} 

ipfiXt-uY? i; ), 


n + 0 , 

n = 0 , 


Y n+t , n < -t, n > 1 

(X t _i +n ,y„ +t ), -t + 1 < n < 0 . 


• X t = X 0 oa t = ipf (Xt-uYt) = <p? y/* 1] ), 

A standard universal dilation is therefore specified by the term (G z , £/, a, (Q{p}){p}ep), 
where (G Z ,Q) represents the environment state space, a the one-step evolution of the system 
with the environment, and Q{P} is a family of initial distributions for the environment state 
which give rise to the corresponding Markov evolutions for the system. 

With a standard universal dilation the evolution of every global random variable F £ C°°(F) 
is described by the group of ^-automorphisms F i—> F o of. 

Note that a standard universal dilation always allows to see every Markov chain also as an 
automaton with independent random inputs Yj, which now are provided by the environment via 
the dynamics a. 

A standard universal dilation is not uniquely determined by the state space E, but it always 
exists. 


Theorem 1. For every finite state space E, there exists a standard universal dilation 
(G z , G, a, {Q{p}){p}ev) of the Markov evolutions on E. 

Proof. We have to exhibit a proper set G, together with the coupling ipi and the probability 
measures 

Given E = {1,..., N} and the set L labelling the all possible maps (3 : E E, we set 

G = E x L, i,j,k £ E, £ £ L, g = {j,£)eG. 

Now, taken two points ( i , (1,^)) (i\ (1,/)) in E x G, we get (/3e(i), (i,£)) (f3e(i'), (* / ,^ / )) 

and so we can find an invertible map 


ip : E x G —> E x G, 


<p{h = 


(A>(*)> M))> 


if 3 = 1, 
if 3 + 1- 


( 8 ) 


The coupling ip\ is defined to be ip when G = G±. 

Given {P}, we choose a decomposition @ for every P(t), thus obtaining the distributions q(t) 
on P; hxed an arbitrary state in E, say 1, for every distribution q on L we define the distribution 
q = Si<S>q on G , that is q g = qu % t) = Sj t i q£. Fixed also an arbitrary distribution Q o for X n < 0 G n , 
we define Q{py = Q o < 8 > (<S>teN on & and, given k £ E, we define P^qp} = 5k < 8 > Q{p} on F, 
that is 

m 

Pfc,{P} ^^0 = h Y- m = C/—mi ■ ■ ■ j Y m = gm'j = $i,k Qo(.S—mi ■ ■ ■ > ffO ) Qg n (^)> V?7l > 1. 

n= 1 

Then Xq = k a.s. and To has distribution Q{py, so that (Y n )n<o ~ Qo while Y n ~ q{n) for 
every n > 1 and they are all independent. Of course, if we change k and {P}, then Ao ~ 5k 
for every {P} and T 0 ~ Q{ P y for every k. Moreover, every (£l,F, (F t ) t > o, (X t )t>o, (Pfc,{p})feep) 
is a Markov chain with transition matrices {P}, independently of Qo and of the definition of 
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ip(i, (j, £)) for j 7 ^ 1. Indeed, for every t > 1 and every event (Y t = gt, ..., Yi = gi, Xo = *q) 
with positive probability P fc rpi, that implies *o = k and g s = (l,i s ) f° r all 1 < s < t, it holds 


k,{P} (X t+1 = j | Y t = (1, i t ), ...,Y 1 = (1, i x ),X Q = k) 

= J2 ¥ k,{P}(x t+1 =j\Y t+1 = (l,£),Y t = {l,it),-- .,Y X = (l,ii),X 0 = k) 
ecL 

X p fc>{P} (Y m = (1 ,i)\Y t = (i,€ t ),...,ri = (i,ii),x 0 = k) 

= ^2i^Pt°l3z t o-of3 tl {k),j = (IjO) 


£gP 


- S Pe(fa t °-° &i( fc ))>J qe ^ + 

i&L 

= P/3e t o—o/3 il (k)j(^ + 1 ), 

where the last equality follows from © and © • Thus 


^k,{P}( x t+ 1 =j\X t ) =p Xtj {t + 1) =F k (X t+1 = j\X t ), \/{P},k,j,t. 


Therefore ( G z ,G,a , (Q{p}){p}g-p) is a standard universal dilation of the Markov evolutions on 

E. □ 


In this construction, even if each g E G has two components, g = ( j,i ), the probability is 
always concentrated only on those g of the kind g = ( 1,4 but we need the first component j 
to define an invertible ip. Analogously, we are considering the evolution only for positive times 
so that all the components g n , n < 0 , are never involved in the interaction with the system, but 
they are needed to define an invertible shift 


Example 1. As a first example, let us characterize the dilation provided by Theorem in 
the degenerate case of a homogeneous deterministic evolution. Let every P(t) be equal to the 
deterministic matrix Dg associated to the map /3g on E. Then, for positive t, the distributions 
q(t) must equal Sg, so that q(t) = 5\ ® 5g = b(\j\ and the environment components Y t , the ones 
which will interact with the system, turn out to be deterministic. For the sake of simplicity, let 
us choose the same degenerate distributions also for all t < 0 , that is Qo = ( 8 >«o an d, f° r 
every system initial state k, let ^ k ,n f = &k <8 > (®z<Vl/)) denote the corresponding probability 
on T. Then, under every P k ,Dji the environment initial state To itself is deterministic and, for 
every coupling <p satisfying JHJ), the following equalities hold almost surely 


Xq = k, T 0 = ((M) n ), z o = (k, (1,4), 


Z t = a t oZ 0 = (f3i(k),(g n ) n ), Vf > 0, where g n 


Xt = fy t (k). 


( 1 , i), n < —t, n > 1 , 

((3f- 1+n (k),i), -t + l<n<0, 


Note that, even if Dg is a permutation, that is even if /3g is invertible (and no dilation would 
be needed), the dilation provided by Theorem |T| is not trivial at all. Indeed, just because of 
the universality of the construction, a dilation (G z , Q, a, Q{ P }) is usually non minimal for a 
particular evolution {P}. 





Example 2. As a second example, let us consider a homogeneous evolution in E = {1,2}, say 
P(t) = P for every t. In this case there are four deterministic matrices 


D i = 



D 2 = 



d 3 = 



and we decompose the stochastic matrix P as 


D 4 = 


p = r * 11 Pl2 \ =P 11 P 21 D 1 +P 11 P 22 D 2 +P 12 P 21 -D 3 +P 12 P 22 A >4 = qi Di + q 2 D 2 + q 3 D 3 + q 4 D 4 . 
\P 21 P 22 J 

Therefore L = {1, 2,3,4} and the environment state space is T = G® with G = {1, 2}x { 1 , 2 ,3,4}. 
The distribution q on L gives the distribution q = di ® q on G and the distribution (££) z q on G z . 
Choosing this latter as the distribution Qp, under every Pp = 5^ < 8 > (<S>z o) the initial states of 
the system and of the environment are 


Xg = k a.s., 


T 0 = (Y n ) n , Y n i.i.d., Y n ~ q, 


so that, for example, P k,p(Y n = (1,3)) = p\ 2 P 2 i- Even if the definition of the coupling ® for 
j / 1 plays no role, let us fix c p setting 


p : E x G -*■ E x G, 


f (&(*)> M))> if 3 = 1) 

¥>(*> ■= l {Pt+2{i), (M)), if j = 2 , i = 1 , 

[(&+i(*)>M))> if J = 2 , i = 2, 


where t + 1 and t + 2 are sums modulo 4. Then, always under P^ p, after one step the states of 
the systems are both random and, precisely, 


Xi = <p? (fc,Ti), 


Ti = (Tj 1 )). 


y(i) 

■ L n 


Y n+ 1 , n < -1, n > 1 

^f(fe,Ti), n = 0, 


where A} = pf(k, Y \) = /%(fc) when 1} = (1,E). Therefore 
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Pfc,p(A"i = j) = Pfc,p(A'i = j|T 1 = (1, £)) Pfc,p(Ti = (1, E)) = ^2 8p e (k),j Qi 

t =\ 1 = 1 £=1 

= Pfcj ■ 

Similar calculations for t > 2 then prove that (0, P, (Pt)t>o, {X t )t>g, (Pfc,p)fcep) is a homoge¬ 
neous Markov chain with transition matrix P. 


Let us remark that, as long as we consider only system random variables neglecting the envi¬ 
ronment, we can avoid the shift 1 ? and, starting from this dilation, we can define a deterministic, 
invertible, but inhomogeneous global evolution p>t which never involves the environment compo¬ 
nents g n for n < 0, but which generates the same Markov evolutions for the system. Consider 
indeed the invertible maps on 

<pt := 0^0 r & t ~ 1 , (p t := p t o ■ ■ ■ o tpi = o a*, t> 1, (9) 

where pt actually acts only on £ x Gt, where it is just the coupling (J5J), while pt actually acts 
only on E x Gi x • ■ • x G[. Then, for every t > 0, we get Xt = Xg o a 1 = Xg o p t , so that the 
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same system evolution is obtained without involving g n for n < 0. In particular the stochastic 
evolution of a system random variable / £ C°°(E) is given by the *-unital homomorphism 

f^jt[f}:=f(X t )=fotp t , t> 0, (10) 

which is injective as <pt is invertible. If E g [f o^] g C°°(E ) is the system random variable 
i i —> f o tp(i,g), then the stochastic evolution (fTTHl satisfies 


jo[f] = f(X o), 


it [f ] — 'y ] jt—i 

g£G 


E g[fotp\ 


I (Y t =g)i 


V/ £ C°°(E), t>l, (11) 


where jt~\ E g [f o p\ are Et-i -measurable and I[Y t =g) are the indicators of the events (Y t = g ). 

And now we could even reduce the sample space D from E x to E x G N , restricting here 
J-, J~t and This alternative construction has its counterpart in Quantum Probability 

and, analogously to what happens in the Quantum framework, one could start with it and 
then recover the whole dilation (G z , Q, a, (<3{P}){P}e , p)> so that the two constructions can be 
considered equivalent descriptions of the same situation. 

To conclude this section, let us add two remarks about the choice of probabilities E differ¬ 
ent from the selected Pfc,{P}) again in the case of the dilation built in Theorem El Firstly, the 
decomposition © is not unique, just as the choice of Qq , so that we could find other proba¬ 
bilities P on T inducing the same Markov evolutions for the system. Secondly, we could also 
consider non-factorized Q , or even non-factorized P, thus obtaining new stochastic processes 
(f2, P, (E t ) t >o, (At)t>o,P), without any Markov property guaranteed, which would depend also 
on the definition of tp for j ^ 1 . 


4 UNIVERSAL DILATION OF MARKOV SEMIGROUPS AND 
DILATIONS OF QUANTUM DYNAMICAL SEMIGROUPS 

We want to compare the dilation built in Theorem El with similar constructions typical in Quan¬ 
tum Probability: given a stochastic matrix P on E, we show that the classical evolution (P t )t>o 
(Classical Markov Semigroup, CMS) can be extended to a quantum evolution (T t ) t >o (Quan¬ 
tum Dynamical Semigroup, QDS) for which we can find a quantum dilation which is itself an 
extension of the classical dilation (G z ,Q,a,Qp). Of course, Qp and P k,p stay for Q{pj and 
Pfc/p} when the sequence {P} is constant. 

Given a complex separable Hilbert space 7i, let us denote its vectors by h, or | h) using 
Dirac’s notation, so that (h’\h) denotes the scalar product (linear in h) and \h')(h\ denotes the 
bounded operator h" e-> (h\h") h' . Let B(7i) be the complex *-algebra of bounded operators 
on H and let S(7i) be the convex set of states on B(H), S(7i ) = jp £ B{TL) p> 0 ,trp = lj, 
with its extreme points (pure states) given by the rank-1 projections \h)(h\, ||/i|| = 1, so that 
for every p in S(7{) there exists an orthonormal basis {hi} such that p = YiiPi \ hi){hi\, pi > 0, 
J2iPi = 1- For every p in S(7i), the term p) is a quantum probability space and tr[ap] is 

the expected value of a £ B(7i). 

A family {oaIagA Q B{7i) of normal operators is commuting if and only if the operators a\ 
admit a common spectral representation, i.e. a classical probability space (D,P, P), a unitary 
operator u : 7i —> L 2 (P) and a family {/a}agA L°°(P) of random variables on D, such that 


a\ = u 1 m f x u, 


VA £ A, 
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where, for every / in L°°(P), rrif denotes the bounded multiplication operator by / on L 2 (F). 
Thanks to u the operators a\ admit a common probabilistic interpretation as the random vari¬ 
ables /a, with a joint distribution which depends on the state p £ S(H): via the spectral 
representation, every p = J2iPi \hi){hi\ defines a probability P p on (11, J-), 


dPp 

dP 


^2Pi \uhi\ 2 , 


and so a joint distribution for {/a}agA and hence for {ua}agA> 

= f ^(/ad • • • > f\ n ) dPp = tr [v(a\ 1 , • • •, a\ n ) p \, 

Jn 

n £ N, {Ai,..., A n } C A, rj : C n —► R bounded and continuous. 

An arbitrary one-step evolution in B(H) is given by a stochastic map T, that is a completely 
positive, identity preserving, linear operator T : 13(H) —> 13(H). A stochastic map T is invertible, 
i.e. there exists another stochastic map T' such that T o T' = T' o T = , if an( i only if 

T[a] = «*au with u unitary operator on H (T ^-automorphism of 13(H)). A homogeneous 
evolution in discrete time is given by a QDS, that is a semigroup (T f ) t >o with T as above. 

Let us now give first the definition of Quantum Stochastic Flow (QSF), then the definitions 
of dilation of a stochastic map and of a QDS, and finally the definitions of extension of a CMS 
and of a classical standard dilation. 

Given two Hilbert spaces H and K, and a trace class operator u on /C, the conditional 
expectation of an operator A in B(H ®1C) = B(H) < 8 > B(1C) with respect to a is the operator 
E ct [A] in B(H) defined by 


tr 


E a [A\- p = tr [A • p 0 <j\. 


Vp £ S(H). 


Given H and infinitely many copies 3n of a Hilbert space 3, introduced the increasing 
sequence of algebras 

t 

B m =B(H)®B((g) 3n), i>l, 

n= 1 

a QSF on B(H) in discrete time is a family of *-unital homomorphisms jp : B(H) —> B^b, t > 0, 
induced by a *-unital homomorphism v : B(H) ^ B(H 0 3) through the equations 


Jo 


= Id 


B(H) > 


jt[a] = 


,Jt -1 


E 


\z){z'\\ 


v(a)] <S> \z')(z\, t > 1, 


( 12 ) 


where (|^)} is a given basis of 3 - 

Given a Hilbert space 3, a unitary operator V on H <S> 3 and a state a £ 5(3), the term 

(3, V, a) is a dilation of a stochastic map T on B(H) if 


T[a] = E ct [V* • a <8> I 3 • V], Va £ B(H), 


so that the one-step evolution T of the quantum system H is represented in terms of an invertible 
evolution V of the system coupled with an environment 3 with initial state a. A dilation always 
exists, not unique, and it can always be chosen with a pure state o = \(j)){(j)\, 4> £ 3 - 

Given a Hilbert space /C, a unitary operator U on H <g> K, and a state a £ 5(/C), the term 
(/C, U, cr) is a dilation of a QDS T l on B(H) by the group of ^-automorphisms A U^AU 1 
on B(H) <g> B(K) if 

T 4 [a] = E a [U** • a® I 3 • U f ], Va £ 13(H), t > 0, 
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so that the discrete time evolution T* of the quantum system H is represented in terms of an 
invertible homogeneous evolution 17* of the system coupled with an environment /C with initial 
state a. 

Given a Hilbert space 3, a unitary operator V on H <8> 3 and a unit vector in 3, the term 
(3,V, 4>) gives a dilation of a QDS T* on B(H) by the QSF (jt)t> o induced by the *-unital 
homomorphism 6 : 13(H) —> B(H <g> 3), v[a\ = V* ■ a ® % • V if 

T*[a] = [ j t [a] ] , Va G B(H), t > 0, 

so that the discrete time evolution T l of the quantum system H is represented as the expectation 
of a QSF associated to an invertible coupling V, analogously to the stochastic evolution m, 

CD- 

A typical construction employs the same dilation (3, V, (j>) of a stochastic map T to dilate the 
QDS T f by the associated QSF and, at the same time, by a related group of ^-automorphisms. 
Taken infinitely many copies 3 n of 3, n G Z, let us define the Hilbert space K. = ® neZ 3n, 
infinite tensor product ( [10] Ex. 15.10 or [5] Ex. 11.5.29) with respect to a stabilizing sequence 
of norm-1 vectors i/j n G 3 n such that i\i n = cj> for every n > 1. Let us set T = 0 neZ ^ n , norm-1 
vector in 1C. Let us also embed each £>( 0 ) t] into B(H) < 8 > B(/C) by tensorizing with the identity. 
In particular let us denote by V n the unitary operator V on H < 8 > 3n ; let us identify all of them 
with their extension to H<S>1C, and, for every t > 1, let us define Vt = Vf ■ - V i, unitary operator 
acting on H <g> 3i <8> ■ ■ ■ < 8 > 3t- Then j t [a] := V* ■ a < 8 > 11*; • Vt is the QSF satisfying 


Jt[a\ 




E 


\z){z'\ 


I [V* ■ a < 8 ) % • V\ ®\z')(z\, t> 1, 


for every basis (|z)} in 3, and it dilates the QDS T f : 

T l [a] = E* [. j t [a \], Va G B(H), t > 0. 


Moreover, introduced the shift operator 0 : K. —> K. 


n ^ A., Z n G 3 n 1 " ^n+1 £ ^n+l £ 3n (13) 

(where, of course, z n = cf> for large n), extended it to a unitary operator @ : H ® 1C H ® 1C, 
let us define the unitary operator 

17 = 0 Vi. (14) 

Then (/C, U, T) defines a group of ^-automorphisms of B(H) ® B(1C) which dilates the QDS T l : 

T*[a] = Em [U* t ■ a ® ■ U 1 ], Va G 13(H), t > 0. 

Of course U* 1 ■ a (8) 11/c • = jt[a] for every a £ B(H) and t > 0, as 0 commutes with a <8> Hyc- 

Given the state space E, we say that a QDS T* on some B(H) extends a CMS P l on C°°(E) 
if 


• there is a ^-isomorphism / between C°°(E) and a (commutative) subalgebra of 

B(H) such that 

r[m / ]=m P/ , V/ G C°°(E). 

• for every k G E there exists a state pk G 5(77) such that 


^/(fc) 


tr 


m 


/] Pfc 


V/ G £°°(£), t > 0. 


(15) 
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Such extension always exists. For example, if we take H = C N = with its canonical basis 
{K)}ie£ and we embed C°°(E) in 13(H) by 

f ^m f = ^2f(i)\i)(i\, (16) 

i&E 

♦-isomorphism between C°°(E) and the subalgebra of the diagonal operators V(H) C B(H), 
then, using decomposition m and notations Q, a stochastic matrix P is extended by 

T[ a ]='52qe\i)(Pe(i)\a\Pe(i))(i\, Va£B(H), (17) 

eeL 

i£E 

and equation os holds if every system state k G E is associated to the pure state p} ; = |A:)(A:|. 
Note that this extension maps B(H) to T>(H) in only one step. The extension of a CMS is not 
unique at all: for example, with the same embedding m and the same choice of the states pk, 
every permutation P can be extended also by a ♦-automorphism T[a] = u*au , provided that, 
for every j in E , u \ j) = P* \j) up to a phase factor (necessary and sufficient condition). 

Given a CMS P t on C°°(E) and a quantum extension T l on some B(H ), we say that a 
dilation ( IC,U,ct ) of T* extends a standard dilation (G z ,G,a,Qp) of P l if 

• there exists a ♦-isomorphism F i—> mj? between = C°°(E) <g> (g) nGZ £°°(G n )^ and a 

(commutative) subalgebra of B(H <8> 1C) such that 

U m F fT* = m Foa t, VF E Cf , f E Z 

• for every k E E there exists a state pk E <S(H) such that 

Efc,p[r/(Fi,... ,F n )\ = tr [r)(m Fl ,... ,m Fn ) p k <g> a], (18) 

for every F\,... ,F n G CP£ and every rj : C n —»• R bounded and continuous. 

Note that the spatial tensor product of C'*-algebras Tg 3 ( [5] §11.4) is a proper sub C'*-algebra of 
C°°(F) (it consists of all continuous functions on the compact set E x G z ) and that it is invariant 
for the evolution F i—> F o a*. Therefore also its image under F i—> m F has to be an abelian sub 
C*-algebra of B(H®K), invariant for the group of ♦-automorphisms A i—> U* t AU t , which hence 
extends the classical evolution. Moreover, the states Qp and a of the environments, together 
with the dynamics a and U, give rise to the same joint distribution for the trajectories of the 
global random variables F G C,^ and for the trajectories of the corresponding normal operators 
m F G B(H<S>1C). This happens for every possible starting state k of the classical Markov system 
and for the corresponding state pk of its quantum counterpart. 

Theorem 2. Let P be a stochastic matrix on a finite state space E and let (G z ,Q,a,Qp) be 
the dilation provided by Theorem Q with Qp = Then there exist a Hilbert space H, a 

stochastic map T on B(H) extending P, and a dilation (3, V. <p) of T such that the associated 
dilation (<2>z 3, U, 4>) o/T* extends (G Z ,G, a, (£)%([). 

Proof. As in Theorem El let E = {1,..., IV}, let Dp = Y^ieE |/%(®))(*l deterministic 

matrices corresponding to the deterministic evolutions fin : E —> E, as in Eq. ©, labelled by 
£ G L. Then fix a probability qn on L such that P = as in Eq. Q, and set G = E x L. 

Now, taken H = with its basis {| i)}ieE, embedded C°°(E) in V(H) C B(H) by (ITHll . 
let P be extended by the stochastic map T (fITI) . Taken 3 = Cl G ’l = <C) ExL \ with its basis 
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{| j, l)}j£E,i£L, let V be the unitary operator on H ®3 which equals the adjoint of the permutation 
matrix associated to the coupling p J5J), with respect to the basis {|i, j, fy}i,jeE/eL : 

V = 12 I 

i,j £E ,€ g .£/ 


Taken the pure state 4> = l n 3, we are going to show that (3 ,V,(j>) is a dilation 

of T enjoying the required properties. 

For every a £ 13(H) and every p £ S(H ), p = J2i,i'£E Pa 1 K)(^l> 


tr 


V* -a®t 3 -V ■ p® |0)(0| 


V<uV<iPPii' (*',1,^1 v* • a ® 1 3 • V\i, 1 ,£) 

i,i f £E 
e,e'eL 


= M^(M0)l a ® %Kl(M))) 

i,i'eE 

£,£'eL 


= \f&VwPii' (f3i>(i'),U'\a® mp e (i),l,e) 

i,i'g E 
e,e'eL 


= ^2 Pa (fa(i)\a\Pe(i)) = 

ieE,lGL 


tr 


T[a\p 


so that (3, V, i j>) dilates T. 

Let now (IC,U, T) be the associated dilation of T l given by K. = ® neZ 3 n with respect to 
the stabilizing sequence ipn = 4*1 by the unitary operator U defined by Eq. m and m and 
by the pure state \k = (SlnezV’n- I 11 order to show that (lC,U,^f) extends (G z , Q, a, q), let 
P(3n) denotes the subalgebra of B( 3 n ) consisting of diagonal operators with respect to the basis 
{| jn,£n)}j n eE,e„eL- Then let F *—> thf be the ^-isomorphism between C^ and the spatial tensor 

product T>z = T>(H) ® ( ^(3n)) defined by mapping each 


FeC 


OO 

[— m,m\ 


m 

C°°(E) ® ( 0 £°°(G n )) 

n——m 


/ m \ 

C°°(ex( X G n )), 

' n=—m ' 


m > 1 , 


to the operator 


UlF = , 

i,j_ rn ,...,jm^E 


^ F{i\ j—m 5 ^—m\ • • • 5 jmj An) | A j—my ^—m] • • • 5 jmi An) (A j—my ^—m] • • • 5 jmy 


belonging to P(W) <g> ( <g)™ = _ m £>(3n)) C 5 (ft (8) (0£L_ m 3n))- For every F belonging to an 
algebra F^ mH , 

0* mj? 0 = mpo’d, V*mFV = rriFoip, U*rriFU = mFoa, 
so that, by continuity, 


U* t m F U t = m Foa t, VFeF^, feZ. 

Finally let us consider the joint distribution of trajectories and let us show that property 
holds if every system state k £ E is associated to the pure state pk = |fc)(fc| on 13(H). 
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Since rj{F \,..., F n ) belongs to C % whenever F\, . . . , F n all belong to CF£ and r\ is bounded and 
continuous, it is to be checked only that 


E k ,p[F] = tr [m F • \k)(k\ $ |T)(T|J , VF £ 
If F belongs to then 


tr 


m F ■ \k)(k\ <g> |W)(^| = F(k]j- m ,£- m ;...;j m ,£ m ) (j n ,4|</ 

n= — m 

m 

— ^ ] F{k] ] jm^m) ^(jn/n) = ^ , k,p[F], 




and the general case follows by continuity. 


□ 
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